Real-time Driver Drowsiness Detection with Android

Abstract

Major causes for the road accidents are Driver Fatigue and Distraction. Recently many driver monitoring systems have been proposed for monitoring driver activities to avoid accidents. Almost all the systems which exists till date are in the specialized embedded or hardware form, which are present in expensive and luxurious cars. This research paper presents a cost efficient approach to detect driver fatigue and monitor dizziness/drowsiness in real-time. An intelligent android app to monitor driver fatigue and distraction during travel using Adaptive Template Matching and Adaptive Boosting is achieved and implemented. Developing of technologies for preventing drowsiness is quite challenging in the area of accident avoiding systems. Preventing drowsiness during driving requires a method to accurately detect lowering of driver attentiveness and a method for alerting and to forewarn the driver. In this detection approach, we developed an android app that utilizes image processing technology and analyse all the images captured through the user’s phone camera. Drop of alertness is detected based on the degree to which the driver\'s eyes are open or closed. This application provides a non-contact method for judging various levels of driver awareness and enables an early detection of a reduction in attentiveness while driving.

Introduction

I.  NTRODUCTION

Continuous driving for long hours often leads driver’s mental fatigue and drowsiness [1]. According to many researches drowsiness is related to thousands of traffic accidents each year, the accidents produce approximately 50% of death or serious injuries. Sleep keeps us healthy and functioning well. It lets human body and brain repair, restore, and reenergize. It is one of the basic needs of the human being, lack of sleep causes the body to react inefficiently, reducing reaction time, also produce low alertness and distraction which leads to inability to perform activities related to driving a car. Therefore, it is very much needed to monitor the drowsiness level of the driver and alert him through alarm when required.

These days, Development in technological field been increased speedily [2]. There is a continuous improvement in the sector of managing digital image detection with the help of a camera sensors. Face detection technology can be easily achieved anytime and anywhere. The primary thing required for face detection is to recognize facial expressions and locate coordinates of eyes, ears and nose, this will improve the detection accuracy [3]. In India, 40% of accidents and crashes on highways occur due to drivers being drowsy while long driving hours. When a person is sleep deprived, Drowsiness can affect him in workplace especially when driving heavy duty trucks or goods carrying vehicles. This research paper is based on the implementation of system which will track the driver in real time to reduce the fatalities on highways.

Feasibility plays an important role in any application’s development life cycle, hence to balance the feasibility and make this android application available to all the end-users in an efficient manner and cost-effective manner remains the top priority.

This proposed android application can be used by any driver with just getting a place holder for android phone and application pre-installed in it.

As per the reports of surveys which suggests an intelligent and cost efficient approach to be implemented to encounter the issue there is a huge scope the world using this cost effective and accurate approach.

This user-friendly and easy to use application doesn’t need any kind of external sensor or Embedded system to use it. It will only need an android device and a mobile place holder. It can be beneficial for CMV ‘s (Commercial Motor Vehicle) as well as all other kind of vehicles. This application is portable enough to run on any device which runs on Android operating system.

As we have initially developed this Real-time Driver Drowsiness Detection Android Application for the Android users to detect face and eyes closure. In Future a same application for iPhone users can also be developed which will also result in high accuracy of the application by making it worldwide available to all the Mobile OS. In near future, extraction of features from face as well as posture of body in real-time while driving.

II. LITERATURE SURVEY

Some existing work has been done on face-detection and eye-detection in real-time using machine learning techniques. In this survey, we are focusing on existing researches and studies related to driver drowsiness issues with mobile based approaches.

R. Manoharan and S. Chandrakala [4] have taken some initial efforts on Android Auto based driver monitoring system. They have implemented the real-time fatigue monitoring system which uses Haar classifier object detection to detect face and facial movements with adaptive template matching. This system is compatible with android mobile as well as with android auto with some modifications.

The frameworks based on PERCLOS follow the pipeline of eye and face-detection. One of the first PERCLOS based research on drowsiness detection was published by Carnegie Mellon Driving Research Center by Grace et al [5]. They have obtained results from linear regression techniques.

In the next paper by Singh H, Bhatia JS, Kaur J [6] they have implemented a system that is based on eye movements to detect the fatigue of driver it will warn the driver after half sec while keeping driver’s record of eye movements through whole journey. They have proposed an algorithm which is faster than PERCLOS which makes the system respond within half second.

In the research article by W. W. Wierwille, S. S. Wreggit, C. L. Kirn, L. A. Ellsworth, R. J. Fairbanks III [7], the distinguishing feature of their proposed algorithm is that the method uses 17 facial features based on movement of facial muscles. Additionally, they have attached the filter which differentiate between smile and drowsiness.

The partially visible face can be detected and recognised with very few facial features in real time. This method gives an efficient and rapid approach for detecting or recognizing the occluded human face. This method has been proposed by Ashish Kumar and Dr. P. Shanmugavadivu [8].

The results collected by testing the algorithm proposed by Ahmad Zakasi, Huda Ubaya, Yougi Sanjaya, Desris Stiwant [9] in their research article is displayed in intact, either the side of face or the attribute of facial features. Image capturing is done in 320 x 240 pixels.

This algorithm will successfully detect if there’s a face or not in the picture.

In the Research paper by Panagiotis Drakopoulos, George Alex Koulieris [10], they have proposed detection of iris and tracking of pupil in their article. It is based on Virtual Reality (VR) and works in smart phones having the front facing camera with no new modifications in the phone. The proposed method of this eye tracking technique indicates the enough accuracy and precision in the results.

The approach has been created to track and detect the pupil in an energy-controlled way. Based on results on pupil movement and three dimensional eye model, it can also precisely estimate the FOV (field of view) gazed by eye. The average error in the outcome can be lessen to half approximately in the range of 0.3 to 0.5 degree by using the method proposed by Xiaokun Li and William G.Wee [11].

With the help of Matlab and Microprocessor, an approach to discover the drowsiness of the person who is driving the vehicle has been proposed by Zeeshan Ali Haq and Ziaul Hasan in their research paper [12]. The results of this implemented system give 97% and 87% of positive outcomes for eye detection and eye blinking of the candidate.

A robust and real-time monitoring system to monitor the losing of attention of vehicle drivers has been proposed by Anirban, Anjith, S.L.Happy and Aurobinda in their research papers. Haar classifier is used to detect face in this approach. Bihistogram equalization has performed to compensate the effect of variation. This algorithm provides results in quite good speed of 9.5 fps which correctly estimates the state of eye accurately in real-time.

Wang Yang and Zheng Jiachun’s [13] proposed method for face detection based on YOLO (you only see once) version 3 which has shorter detection time compared to traditional algorithm.

This method can reduce the rate of error and miss rate. It assures high test rate in any environment which makes it possible to give outcomes in real-time situations.

The region of eye is used to identify and differentiate between different individuals. The purpose of this system was to verify if it efficiently recognizes different faces or not with only using the small region of face as input, this method based on facial features extraction was implemented by Muhammad Younus Javed and Sajjad Moshin [14].

The paper presented by Manu B.N [15] proposes the implementation of drowsiness detection which performs well under various lighting conditions. With the accuracy of 94.58%, this method uses correlation coefficient template matching which makes it faster comparatively recent methods. Although their system did not work well when head is tilted right or left.

III. PROPOSED SYSTEM

1. Real-time Input Video: In this phase the Application asks for the camera Permissions and when allowed by user it uses the Camera of the android smartphone and initializes the processing.
2. Face Detection: In this phase the Human Face is firstly defected using the Face detector & Face detector builder classes of Google vision API and then it proceeds to the eye detection. If the Face is not visible it goes back to the phase 1
3.  Eyes Detection: In this Phase the Eyes of the user are detected using the Contour class and if they happened to be closed it proceeds to the next step that us eye status analyzing if the Eyes are closed it goes back to the previous step.
4. Eye Status Analyzing: This Phase determines that the eyes are closed or not using the time sensitivity set by the user (0-8 sec) if it happens to be found closed then it proceeds to the next step i.e. Alarm beeping and if it is open it goes back to the previous step 1.
5. Alarm Beeping: This is the Final phase of the architecture workflow If the eyes found closed then it immediately Alerts user by beeping the alarm and status message: Alert

???????A. Google vision API Classes:

1. Contour: A set of points that outlines a facial landmark or region such as eye, face, or lips. When 'left' and 'right' are used, they are relative to the subject. For example, the LEFT_EYE contour is the subject's left eye, not the eye that is on the left when viewing the image.
2. Face: A human face detected in an image or video. It is important to note that all fields described here are with regards to the image that the detector has processed. Many live apps that process images directly from the camera show the user a mirrored display of the actual image.
3. FaceDetector: Detector for finding Faces in a supplied image. A face detector is created via an associated builder class, specifying the relevant detection options. For example, the code below creates a face detector which is optimized for tracking a single, relatively large face (e.g., the user of a device).
4. FaceDetector.Builder: Builder for creating face detector instances.
5. Landmark: A point on a detected face, such as an eye, nose, or mouth. When 'left' and 'right' are used, they are relative to the subject. For example, the LEFT_EYE landmark is the subject's left eye, not the eye that is on the left when viewing the image.
6. LargestFaceFocusingProcessor: Face processor that focuses on tracking a single "prominent face", in conjunction with the associated FaceDetector. A prominent face is defined as a face which was initially the largest, most central face when tracking began. This face will continue to be tracked as the prominent face for as long as it is visible, even when it is not the largest face.
7. LargestFaceFocusingProcessor.Builder: Builder for creating a LargestFaceFocusingProcessor.

IV. OBSERVATIONS AND RESULTS

Above are the implementation snapshots of Real-time Driver Drowsiness Detection Android application.

In the figure 1(a) it shows that the face is visible and the eyes are also opened.

In the figure 1(b) it shows that the face is visible but the eyes are closed so it is alerting the driver.

In the figure 2(a) it shows that the face is not visible and hence the status is saying Face missing.

In the figure 2(b) it shows that when the night mode is on during night and if the driver's eyes are closed ut alerts them

V. COMPARISON ANALYSIS OF EXISTING SYSTEM WITH OUR SYSTEM

VI. ACKNOWLEDGEMENT

It gives us great pleasure and satisfaction in presenting the final year project research paper on real time driver drowsiness detection Android application. We  have further more to thank our guides Dr. Jayashree Pansare and Computer Department HOD Dr. N. F. Shaikh to encourage us to go ahead and for continuous guidance. We would like to thank all those, who have directly or indirectly helped us during of the work which made this project possible.

Conclusion

In order to reduce the number of accidents due to the driver’s drowsiness. This research paper proposes a real-time Drowsiness Detection Android Application. This Application offers easy flexibility from Android device to Android Auto with few modifications. No IR illuminators are used in this project due to which this system will not provide prime results under low light/no light conditions. In near future, to increase the efficiency and accuracy of this application, the usage of facial expressions as well as the body posture while driving will be inspected.

References

[1] D. F. Dinges, “An overview of sleepiness and accidents,” J. Sleep Res., vol. 4, no. s2, pp. 4–14, Dec. 1995. [2] M. T. Tombeng, A. A. Taghulihi, and J. M. S. Waworundeng, “Android Based Application To Detect Drowsiness When Driving Vehicle,” [2019 1st International Conference on Cybernetics and Intelligent System (ICORIS)] , August 2019. [3] I. Setyawan, I. K. Timotius, and A. A. Febrianto, “Frontal face detection using Haar Wavelet Coefficients and Local Histogram Correlation,” Journal of ICT Research and Application, vol. 5, pp.157- 172, 2011. [4] Manoharan, R.; Chandrakala, S. (2015). International Conference on Computing and Communications Technologies (ICCCT) - Android OpenCV based effective driver fatigue and distraction monitoring system. , (), 262–266. [5] R. Grace et al., “A drowsy driver detection system for heavy vehicles,” in Proc. 17th DASC AIAA/IEEE/SAE Digit. Avionics Syst. Conf., vol. 2, Oct./Nov. 1998, pp. I36/1–I36/8. [6] Singh H, Bhatia JS, Kaur J.: Eye tracking based driver fatigue monitoring and warning system. InPower Electronics (IICPE), 2010 India International Conference. pp. 1-6. IEEE (2011) [7] W. W. Wierwille, S. S. Wreggit, C. L. Kirn, L. A. Ellsworth, R. J. Fairbanks III, “Research on vehicle-based driver status/performance monitoring: development, validation, and refinement of algorithms for detection of driver drowsiness”, National Highway Traffic Safety Administration, U.S. DOT Tech Report No. DOT HS 808 247, 1994. [8] “Rapid Face Detection and Annotation with Loosely Face Geometry” by Ashish Kumar and Dr. P. Shanmugavadivu at 2016 2nd International Conference on Contemporary Computing and Informatics. [9] Zarkasi, Ahmad; Nurmaini, Siti; Stiawan, Deris; Firdaus, ; Ubaya, Huda; Sanjaya, Yogie; Kunang, Yesi Novaria (2018). [IEEE 2018 International Conference on Electrical Engineering and Computer Science (ICECOS) - PANGKAL PINANG, Indonesia (2018.10.2-2018.10.4)] 2018 International Conference on Electrical Engineering and Computer Science (ICECOS) - Face Movement Detection Using Template Matching. , (), 333–338. doi:10.1109/ICECOS.2018.8605215 [10] Drakopoulos, Panagiotis; Koulieris, George Alex; Mania, Katerina (2020). [IEEE 2020 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) - Atlanta, GA, USA (2020.3.22-2020.3.26)] [11] Li, Xiaokun; Wee, William G. (2009).2009 16th IEEE International Conference on Image Processing (ICIP) - An efficient method for eye tracking and eye-gazed FOV estimation. , (), 2597–2600. [12] Haq, Zeeshan Ali; Hasan, Ziaul (2016) “Eye-blink rate detection for fatigue determination.” [IEEE 2016] - Delhi, India [13] Yang, Wang; Jiachun, Zheng (2018). 1st IEEE International Conference on Knowledge Innovation and Invention (ICKII) - Real-time face detection based on YOLO. , (), 221–224. [14] Muhammad Sharif, Muhammad Younas Javed and Sajjad Mohsin (2012) “Face Recognition Based on Facial Features” -COMSATS Institute of Information Technology [15] Manu, B. N. (2016). [IEEE 2016 12th International Conference on Innovations in Information Technology (IIT) - Al-Ain, United Arab Emirates (2016.11.28-2016.11.30)] 2016 12th International Conference on Innovations in Information Technology (IIT) - Facial features monitoring for real time drowsiness detection.

Copyright

Copyright © 2022 Dr. Jayshree Pansare, Ranjeet Deshpande, Shubham Shingare, Harshal Deokar, Piyush Manwar. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.